Mean Average Precision Research Articles

A Comparative Dataset of Annotated Broccoli Heads Recorded with Depth Cameras from a Moving Vehicle

An extensive, publicly available dataset is presented—the LAR Broccoli dataset—which contains 20,000 manually annotated images of broccoli heads captured from a moving tractor at an organic farm in the UK. The dataset contains images of the same row of broccoli heads recorded at 30 frames per second (fps) with three different cameras. Two off-the-shelf, relatively low-cost depth-sensing cameras were used, with the tractor moving at a speed of around 1 km/h, in addition to a webcam, with the tractor moving twice as fast. The utility of the dataset is demonstrated in four ways. First, three different state-of-the-art detector models were trained on the dataset, achieving an overall mean Average Precision (mAP) score of over 95% for the best-performing detector. The results validate the utility of the dataset for the standard task of in-field broccoli head recognition. Second, experiments with transfer learning were conducted, initialised with a smaller pre-trained broccoli detection model, and refined with the LAR Broccoli dataset. Third, we assessed the advantages of transfer learning not only using mAP but also according to time and space requirements for training models, which provides a proxy metric for energy efficiency, a practical consideration for real-world model training. Fourth, the cross-camera generalisation among the three camera systems was compared. The results highlight that testing and training detector models using different camera systems can lead to reduced performance, unless the training set also includes some images captured in the same manner as those in the test set.

A Computer Vision Framework for Structural Analysis of Hand-Drawn Engineering Sketches

Structural engineers are often required to draw two-dimensional engineering sketches for quick structural analysis, either by hand calculation or using analysis software. However, calculation by hand is slow and error-prone, and the manual conversion of a hand-drawn sketch into a virtual model is tedious and time-consuming. This paper presents a complete and autonomous framework for converting a hand-drawn engineering sketch into an analyzed structural model using a camera and computer vision. In this framework, a computer vision object detection stage initially extracts information about the raw features in the image of the beam diagram. Next, a computer vision number-reading model transcribes any handwritten numerals appearing in the image. Then, feature association models are applied to characterize the relationships among the detected features in order to build a comprehensive structural model. Finally, the structural model generated is analyzed using OpenSees. In the system presented, the object detection model achieves a mean average precision of 99.1%, the number-reading model achieves an accuracy of 99.0%, and the models in the feature association stage achieve accuracies ranging from 95.1% to 99.5%. Overall, the tool analyzes 45.0% of images entirely correctly and the remaining 55.0% of images partially correctly. The proposed framework holds promise for other types of structural sketches, such as trusses and frames. Moreover, it can be a valuable tool for structural engineers that is capable of improving the efficiency, safety, and sustainability of future construction projects.

Multispectral Pedestrian Detection Based on Prior-Saliency Attention and Image Fusion

Detecting pedestrians in varying illumination conditions poses a significant challenge, necessitating the development of innovative solutions. In response to this, we introduce Prior-AttentionNet, a pedestrian detection model featuring a Prior-Attention mechanism. This model leverages the stark contrast between thermal objects and their backgrounds in far-infrared (FIR) images by employing saliency attention derived from FIR images via UNet. However, extracting salient regions of diverse scales from FIR images poses a challenge for saliency attention. To address this, we integrate Simple Linear Iterative Clustering (SLIC) superpixel segmentation, embedding the segmentation feature map as prior knowledge into UNet’s decoding stage for comprehensive end-to-end training and detection. This integration enhances the extraction of focused attention regions, with the synergy of segmentation prior and saliency attention forming the core of Prior-AttentionNet. Moreover, to enrich pedestrian details and contour visibility in low-light conditions, we implement multispectral image fusion. Experimental evaluations were conducted on the KAIST and OTCBVS datasets. Applying Prior-Attention mode to FIR-RGB images significantly improves the delineation and focus on multi-scale pedestrians. Prior-AttentionNet’s general detector demonstrates the capability of detecting pedestrians with minimal computational resources. The ablation studies indicate that the FIR-RGB+ Prior-Attention mode markedly enhances detection robustness over other modes. When compared to conventional multispectral pedestrian detection models, Prior-AttentionNet consistently surpasses them by achieving higher mean average precision and lower miss rates in diverse scenarios, during both day and night.

A Multi-Weapon Detection Using Deep Learning

The escalating usage of light weapons in criminal and terrorist activities has necessitated the development of advanced weapon detection systems. This study centers on the application of the high-speed deep learning detection model, You Only Look Once version 7 (YOLOv7), to address the need for efficient and swift threat identification. The proposed system, trained on a self-curated dataset rich in images of various dangerous light weapons, is designed to recognize and distinguish multiple weapons simultaneously. The unique value of YOLOv7 in object and specifically weapon detection lies in its outstanding speed and accuracy, as demonstrated by certain weapon categories achieving a mean average precision (mAP) of 97%. The system’s performance can potentially be further enhanced by augmenting the image dataset for each weapon category. This study, therefore, not only validates the critical importance of YOLOv7 in advancing detection methodologies but also presents a practical solution for bolstering public safety.  

Enhancing the Safety of Autonomous Vehicles in Adverse Weather by Deep Learning-Based Object Detection

Recognizing and categorizing items in weather-adverse environments poses significant challenges for autonomous vehicles. To improve the robustness of object-detection systems, this paper introduces an innovative approach for detecting objects at different levels by leveraging sensors and deep learning-based solutions within a traffic circle. The suggested approach improves the effectiveness of single-stage object detectors, aiming to advance the performance in perceiving autonomous racing environments and minimizing instances of false detection and low recognition rates. The improved framework is based on the one-stage object-detection model, incorporating multiple lightweight backbones. Additionally, attention mechanisms are integrated to refine the object-detection process further. Our proposed model demonstrates superior performance compared to the state-of-the-art method on the DAWN dataset, achieving a mean average precision (mAP) of 99.1%, surpassing the previous result of 84.7%.

Enabling near-real-time safety glove detection through edge computing and transfer learning: comparative analysis of edge and cloud computing-based methods

PurposeComputer vision and deep learning (DL) methods have been investigated for personal protective equipment (PPE) monitoring and detection for construction workers’ safety. However, it is still challenging to implement automated safety monitoring methods in near real time or in a time-efficient manner in real construction practices. Therefore, this study developed a novel solution to enhance the time efficiency to achieve near-real-time safety glove detection and meanwhile preserve data privacy.Design/methodology/approachThe developed method comprises two primary components: (1) transfer learning methods to detect safety gloves and (2) edge computing to improve time efficiency and data privacy. To compare the developed edge computing-based method with the currently widely used cloud computing-based methods, a comprehensive comparative analysis was conducted from both the implementation and theory perspectives, providing insights into the developed approach’s performance.FindingsThree DL models achieved mean average precision (mAP) scores ranging from 74.92% to 84.31% for safety glove detection. The other two methods by combining object detection and classification achieved mAP as 89.91% for hand detection and 100% for glove classification. From both implementation and theory perspectives, the edge computing-based method detected gloves faster than the cloud computing-based method. The edge computing-based method achieved a detection latency of 36%–68% shorter than the cloud computing-based method in the implementation perspective. The findings highlight edge computing’s potential for near-real-time detection with improved data privacy.Originality/valueThis study implemented and evaluated DL-based safety monitoring methods on different computing infrastructures to investigate their time efficiency. This study contributes to existing knowledge by demonstrating how edge computing can be used with DL models (without sacrificing their performance) to improve PPE-glove monitoring in a time-efficient manner as well as maintain data privacy.

Video surveillance-based multi-task learning with swin transformer for earthwork activity classification

Bulldozers, pivotal in earthworks, traditionally undergo supervision through labor-intensive and potentially unreliable manual methods. This research proposes a vision-based method for automating the monitoring of bulldozer operations. First, this research develops a specialized dataset for deep learning, the bulldozer earthmoving activity dataset. Following this, a novel multi-task video classification network (MTVTNet), the multi-task video transformer network, utilizing a video swin transformer architecture, is proposed. This network is adept at concurrently detecting the shoveling action, state, and soil classification of a bulldozer. The effectiveness of this model is demonstrated through its application in a real-world construction setting, achieving a remarkable 99.68% mean average precision. This method not only facilitates comprehensive automated supervision of bulldozer earthmoving activities but also serves as a valuable data source for assessing the operational efficiency of these machines.

Autonomous surface crack identification for concrete structures based on the you only look once version 5 algorithm

Failure to repair roads in a timely manner may shorten their life and even cause traffic accidents. Thus, accurate crack detection and reasonable classification are crucial for road safety evaluation. In this study, an improved network model based on the You Only Look Once version 5 algorithm is presented, with three additional modules: The first module improves the data processing speed by replacing the C3 module in the original network with a lightweight network model. The second module was used to lighten network weight by reusing a simple convolution structure to equivalently represent the calculation of a convolution layer as a weighted sum of several small convolution blocks. And the third module is used to improve the detection accuracy by removing the upsampling and performing three-way splicing. The proposed model can detect different types of cracks, and an extensive ablation study is reported based on various combinations of the proposed modules. Based on training on a database of 5484 images, the results show that the improved network proposed in this study can effectively identify pavement cracks. Compared with the original network, mean Average Precision is increased by 5.98%, the inference time is reduced by 4.82%, and the model weight is decreased by 17.36%. Additionally, to comply with engineering practice, comparative experiments were conducted on the pre-rotated dataset. The results showed that compared with You Only Look Once version 8, the improved algorithm improved accuracy, recall, average accuracy, and F1 score by 3.28%, 8.46%, 3.79% and 5.89%, respectively. This study can serve as an important reference for the development of crack detection methods.

Enhancing surface drainage mapping in eastern Canada with deep learning applied to LiDAR-derived elevation data.

Agricultural dykelands in Nova Scotia rely heavily on a surface drainage technique called land forming, which is used to alter the topography of fields to improve drainage. The presence of land-formed fields provides useful information to better understand land utilization on these lands vulnerable to rising sea levels. Current field boundaries delineation and classification methods, such as manual digitalization and traditional segmentation techniques, are labour-intensive and often require manual and time-consuming parameter selection. In recent years, deep learning (DL) techniques, including convolutional neural networks and Mask R-CNN, have shown promising results in object recognition, image classification, and segmentation tasks. However, there is a gap in applying these techniques to detecting surface drainage patterns on agricultural fields. This paper develops and tests a Mask R-CNN model for detecting land-formed fields on agricultural dykelands using LiDAR-derived elevation data. Specifically, our approach focuses on identifying groups of pixels as cohesive objects within the imagery, a method that represents a significant advancement over pixel-by-pixel classification techniques. The DL model developed in this study demonstrated a strong overall performance, with a mean Average Precision (mAP) of 0.89 across Intersection over Union (IoU) thresholds from 0.5 to 0.95, indicating its effectiveness in detecting land-formed fields. Results also revealed that 53% of Nova Scotia's dykelands are being used for agricultural purposes and approximately 75% (6924 hectares) of these fields were land-formed. By applying deep learning techniques to LiDAR-derived elevation data, this study offers novel insights into surface drainage mapping, enhancing the capability for precise and efficient agricultural land management in regions vulnerable to environmental changes.

Research on improved YOLOv8n based potato seedling detection in UAV remote sensing images

IntroductionAccurate detection of potato seedlings is crucial for obtaining information on potato seedlings and ultimately increasing potato yield. This study aims to enhance the detection of potato seedlings in drone-captured images through a novel lightweight model.MethodsWe established a dataset of drone-captured images of potato seedlings and proposed the VBGS-YOLOv8n model, an improved version of YOLOv8n. This model employs a lighter VanillaNet as the backbone network in-stead of the original YOLOv8n model. To address the small target features of potato seedlings, we introduced a weighted bidirectional feature pyramid network to replace the path aggregation network, reducing information loss between network layers, facilitating rapid multi-scale feature fusion, and enhancing detection performance. Additionally, we incorporated GSConv and Slim-neck designs at the Neck section to balance accuracy while reducing model complexity. ResultsThe VBGS-YOLOv8n model, with 1,524,943 parameters and 4.2 billion FLOPs, achieves a precision of 97.1%, a mean average precision of 98.4%, and an inference time of 2.0ms. Comparative tests reveal that VBGS-YOLOv8n strikes a balance between detection accuracy, speed, and model efficiency compared to YOLOv8 and other mainstream networks. Specifically, compared to YOLOv8, the model parameters and FLOPs are reduced by 51.7% and 52.8% respectively, while precision and a mean average precision are improved by 1.4% and 0.8% respectively, and the inference time is reduced by 31.0%.DiscussionComparative tests with mainstream models, including YOLOv7, YOLOv5, RetinaNet, and QueryDet, demonstrate that VBGS-YOLOv8n outperforms these models in terms of detection accuracy, speed, and efficiency. The research highlights the effectiveness of VBGS-YOLOv8n in the efficient detection of potato seedlings in drone remote sensing images, providing a valuable reference for subsequent identification and deployment on mobile devices.

Improved object detection method for unmanned driving based on Transformers

Improved object detection method for unmanned driving based on Transformers

Computer Vision-Based Lane Detection and Detection of Vehicle, Traffic Sign, Pedestrian Using YOLOv5

There has been a global increase in the number of vehicles in use, resulting in a higher occurrence of traffic accidents. Advancements in computer vision and deep learning enable vehicles to independently perceive and navigate their environment, making decisions that enhance road safety and reduce traffic accidents. Worldwide accidents can be prevented in both driver-operated and autonomous vehicles by detecting living and inanimate objects such as vehicles, pedestrians, animals, and traffic signs in the environment, as well as identifying lanes and obstacles. In our proposed system, road images are captured using a camera positioned behind the front windshield of the vehicle. Computer vision techniques are employed to detect straight or curved lanes in the captured images. The right and left lanes within the driving area of the vehicle are identified, and the drivable area of the vehicle is highlighted with a different color. To detect traffic signs, pedestrians, cars, and bicycles around the vehicle, we utilize the YOLOv5 model, which is based on Convolutional Neural Networks. We use a combination of study-specific images and the GRAZ dataset in our research. In the object detection study, which involves 10 different objects, we evaluate the performance of five different versions of the YOLOv5 model. Our evaluation metrics include precision, recall, precision-recall curves, F1 score, and mean average precision. The experimental results clearly demonstrate the effectiveness of our proposed lane detection and object detection method.

Performance Evaluation of YOLOv5-based Custom Object Detection Model for Campus-Specific Scenario

This study evaluates the performance of a custom object detection model based on the YOLOv5 architecture, specifically tailored for autonomous electric vehicles. The model undergoes pre-processing using the Roboflow computer vision platform, which offers a wide range of tools for data pre-processing and model training. The experiments were conducted on a diverse dataset comprising various objects encountered in campus-specific driving scenarios, such as pedestrians, vehicles, buildings, and obstacles. The performance of the custom object detection model is assessed using standard metrics, including precision, recall, mean average precision (mAP), and intersection-over-union (IoU) at different thresholds. The training process was conducted in a controlled environment, resulting in a Precision of 0.851, a Recall of 0.831, and a mAP of 0.843. These metrics were analyzed to evaluate the YOLOv5-based custom object detection model's ability to detect and categorize objects accurately, its precision in predicting bounding boxes, and its capability to handle various object categories. We also examined the effects of different hyperparameters and data augmentation techniques on the model's performance, including variations in learning rate, batch size, and optimizer algorithms to determine their impact on accuracy and convergence. This analysis provided valuable insights into the model's strengths and weaknesses, highlighting areas for improvement and optimization. These findings are instrumental in developing and deploying advanced object detection systems to enhance the safety and reliability of autonomous electric vehicles.

Deep learning-assisted automatic quality assessment of concrete surfaces with cracks and bugholes

Cracks and bugholes are common factors affecting the quality of concrete surface, thus automatic detection of apparent defects is essential for concrete surface quality assessment. This study proposes an automatic quality assessment method based on deep learning for concrete surfaces with different scales of cracks and bugholes. In particular, a modified cascade mask region-based convolutional neural network (MC Mask R-CNN) with ResNet and feature pyramid network (FPN) is presented. Furthermore, a novel quantitative analysis algorithm for instance segmentation and size quantification of cracks and bugholes is proposed. Multi-scale features extracted by ResNet can be integrated using FPN. Closed operations and fixed elliptical kernels are adopted to reduce inaccuracies associated with crack discontinuities. Datasets collected at construction sites are used for training and testing of the network after data augmentation. Comparison work was carried out to evaluate the proposed model, and the box mean average precision (APbbox) and the mask mean average precision (APmask) index can reach up to 0.588 and 0.553. The overall quantization errors of cracks and bugholes were within acceptable limits. Validation experiments were conducted to illustrate the validity of the quality assessment method.

YOLOv5s-CEDB: A robust and efficiency Camellia oleifera fruit detection algorithm in complex natural scenes

To solve the problem of poor recognition accuracy caused by various colors, uneven distribution, and occlusion by branches and leaves of Camellia oleifera fruits under natural growth conditions, this study proposes an improved deep learning network, YOLOv5s-CEDB, for Camellia oleifera fruit detection based on YOLOv5s. Coordinate Attention Mechanism (CoordAtt), Deformable Convolution (DConv), and Explicit Visual Center (EVC) are introduced to enhance the network’s local and global feature extraction performance. To improve the detection performance for small and dense targets, the feature fusion module of the network was replaced with the designed light-bidirectional feature pyramid (light-BiFPN) structure. GhostConv was used to reduce the parameters and inference speed of the structure. A dataset with different light conditions, colors, and density levels of Camellia oleifera fruits was established, and performance evaluation experiments were conducted. Experimental results showed that the mean Average Precision (mAP) and F1-score of the designed YOLOv5s-CEDB network reached 91.4 % and 89.6 %, respectively, which were 2.6 % and 3.6 % higher than those of the original YOLOv5s model, respectively, and the influencing frame rate arrived at 37.6 FPS. Under different colors, distribution densities, occlusion scenarios, and light intensities, the detection accuracy of the YOLOv5s-CEDB network model was significantly better than those of the YOLOv5s, YOLOv8s and Faster-RCNN networks. It was verified that the proposed YOLOv5s-CEDB network could significantly improve the accuracy and stability of Camellia oleifera fruit detection, satisfying the requirements of yield estimation and intelligent harvesting.

An Automated Video Analysis System for Retrospective Assessment and Real-Time Monitoring of Endoscopic Procedures (with Video)

Background and Aims: Accurate recognition of endoscopic instruments facilitates quantitative evaluation and quality control of endoscopic procedures. However, no relevant research has been reported. In this study, we aimed to develop a computer-assisted system, EndoAdd, for automated endoscopic surgical video analysis based on our dataset of endoscopic instrument images. Methods: Large training and validation datasets containing 45,143 images of 10 different endoscopic instruments and a test dataset of 18,375 images collected from several medical centers were used in this research. Annotated image frames were used to train the state-of-the-art object detection model, YOLO-v5, to identify the instruments. Based on the frame-level prediction results, we further developed a hidden Markov model to perform video analysis and generate heatmaps to summarize the videos. Results: EndoAdd achieved high accuracy (>97%) on the test dataset for all 10 endoscopic instrument types. The mean average accuracy, precision, recall, and F1-score were 99.1%, 92.0%, 88.8%, and 89.3%, respectively. The area under the curve values exceeded 0.94 for all instrument types. Heatmaps of endoscopic procedures were generated for both retrospective and real-time analyses. Conclusions: We successfully developed an automated endoscopic video analysis system, EndoAdd, which supports retrospective assessment and real-time monitoring. It can be used for data analysis and quality control of endoscopic procedures in clinical practice.

Dual attention-enhanced SSD: A novel deep learning model for object detection

Object detection is a fundamental task in computer vision with significant implications across various applications, including autonomous driving, surveillance, and image understanding. The accurate and efficient detection of objects within images is crucial for enabling machines to interpret visual information and make informed decisions. In this paper, we present an enhanced version of the Single Shot MultiBox Detector (SSD) for object detection, leveraging the concept of dual attention mechanisms. Our proposed approach, named SSD-Dual Attention, integrates dual attention layers into the SSD framework. These dual attention layers are strategically positioned between feature maps and prediction convolutions, enhancing the model's ability to capture informative feature representations across a wide range of object scales and backgrounds. Experimental results on the PASCAL VOC 2007 and 2012 datasets validate the effectiveness of our approach. Notably, SSD-Dual Attention achieves an impressive mean Average Precision (mAP) of 78.1%, surpassing the performance of SSD models enhanced with attention mechanisms such as SSD-ECA, SSD-CBAM, SSD-Non-local attention and SSD-SE attention, as well as the original SSD. These results underscore the enhanced accuracy and precision of our object detection system, marking a substantial advancement in the field of computer vision. Code is available at https://github.com/AlexHunterLeo/Dual-attention-Enhanced-SSD-A-Novel-Deep-Learning-Model-for-Object-Detection

Parameter-efficient framework for surgical action triplet recognition.

Surgical action triplet recognition is a clinically significant yet challenging task. It provides surgeons with detailed information about surgical scenarios, thereby facilitating clinical decision-making. However, the high similarity among action triplets presents a formidable obstacle to recognition. To enhance accuracy, prior methods necessitated the utilization of larger models, thereby incurring a considerable computational burden. We propose a novel framework known as the Lite and Mega Models (LAM). It comprises a CNN-based fully fine-tuned model (LAM-Lite) and a parameter-efficient fine-tuned model based on the foundation model using Transformer architecture (LAM-Mega). Temporal multi-label data augmentation is introduced for extracting robust class-level features. Our study demonstrates that LAM outperforms prior methods across various parameter scales on the CholecT50 dataset. Using fewer tunable parameters, LAM achieves a mean average precision (mAP) of 42.1%, a 3.6% improvement over the previous state of the art. Leveraging effective structural design and robust capabilities of the foundational model, our proposed approach successfully strikes a balance between accuracy and computational efficiency. The source code is accessible at https://github.com/Lycus99/LAM .

Analysis and Comparison of New-Born Calf Standing and Lying Time Based on Deep Learning

Standing and lying are the fundamental behaviours of quadrupedal animals, and the ratio of their durations is a significant indicator of calf health. In this study, we proposed a computer vision method for non-invasively monitoring of calves’ behaviours. Cameras were deployed at four viewpoints to monitor six calves on six consecutive days. YOLOv8n was trained to detect standing and lying calves. Daily behavioural budget was then summarised and analysed based on automatic inference on untrained data. The results show a mean average precision of 0.995 and an average inference speed of 333 frames per second. The maximum error in the estimated daily standing and lying time for a total of 8 calf-days is less than 14 min. Calves with diarrhoea had about 2 h more daily lying time (p < 0.002), 2.65 more daily lying bouts (p < 0.049), and 4.3 min less daily lying bout duration (p = 0.5) compared to healthy calves. The proposed method can help in understanding calves’ health status based on automatically measured standing and lying time, thereby improving their welfare and management on the farm.

PL-DINO: An Improved Transformer-Based Method for Plant Leaf Disease Detection

Agriculture is important for ecology. The early detection and treatment of agricultural crop diseases are meaningful and challenging tasks in agriculture. Currently, the identification of plant diseases relies on manual detection, which has the disadvantages of long operation time and low efficiency, ultimately impacting the crop yield and quality. To overcome these disadvantages, we propose a new object detection method named “Plant Leaf Detection transformer with Improved deNoising anchOr boxes (PL-DINO)”. This method incorporates a Convolutional Block Attention Module (CBAM) into the ResNet50 backbone network. With the assistance of the CBAM block, the representative features can be effectively extracted from leaf images. Next, an EQualization Loss (EQL) is employed to address the problem of class imbalance in the relevant datasets. The proposed PL-DINO is evaluated using the publicly available PlantDoc dataset. Experimental results demonstrate the superiority of PL-DINO over the related advanced approaches. Specifically, PL-DINO achieves a mean average precision of 70.3%, surpassing conventional object detection algorithms such as Faster R-CNN and YOLOv7 for leaf disease detection in natural environments. In brief, PL-DINO offers a practical technology for smart agriculture and ecological monitoring.